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Abstract 

We study causal dynamic approximation of non-bandlimited discrete time processes by 
band-limited discrete time processes such that a part of the historical path of the underlying 
process is approximated in Euclidean norm by the trace of a band-limited process. We 
obtain some conditions of solvability and uniqueness of optimal solution for this problem. 
An unique extrapolation to future times of the optimal approximating band-limited process 
can be interpreted as an optimal forecast. 
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1 Introduction 

We study causal dynamic approximation of non-bandlimited discrete time processes by band- 
limited discrete time processes. This task has many practical applications and was studied 
intensively, mostly in continuous time setting. It is known that it is not possible to find an 
ideal low-pass causal linear time-invariant filter. In continuous time setting, it is known that 
the distance of the set of ideal low-pass filters from the set of all causal filters is positive pQ 
and that the optimal approximation of the ideal low-pass filter is not possible [3]. Our goal 
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is to substitute the solution of these unsolvable problems by solution of an easier problem in 
discrete time setting where the filter is not necessary time invariant. Our motivation is that, for 
some problems, time invariancy for a filter is not crucial. For example, a typical approach to 
forecasting in finance is to approximate the known path of the stock price process by a process 
that has an unique extrapolation. This extrapolation can be used as a forecast. This procedure 
has to be done at current time; it is not required that the same forecasting rule will be applied at 
future times. The present paper suggests to approximate discrete time processes by the discrete 
time band-limited processes. More precisely, we suggest to approximate the known historical 
path of the process by the trace of a band-limited process. In this setting, the approximating 
sequence does not necessary match the underlying process at sampling points. This is different 
from classical sampling approach (see, e.g., [9] , [6] , [7j ) . In |6J-|7J, the estimate of the error norm is 
given. In our setting, it is guaranteed that the approximation generates the error of the minimal 
norm. 

We obtain sufficient conditions of existence and uniqueness of an optimal approximating pro- 
cess. The optimal process is derived in time domain in a form of sine series. To accommodate the 
current flow of observations, the coefficients of these series and have to be changed dynamically. 
The approximating band-limited process can be interpreted as a causal and linear filter that 
is not time invariant. An unique extrapolation to future times of the optimal approximating 
band- limited process can be interpreted as an optimal forecast. 

This paper develops further the approach suggested in [2] , where the continuous time setting 
was considered. We extend now this approach on discrete time processes. 

2 Definitions 

For a Hilbert space H, we denote by (•, the corresponding inner product. We use notation 
sine (x) = sin(x)/x. 

Let Z be the set of all integers, and let Z + be the set of all positive integers. We denote by 
£ r the set of all sequences x = {x(t)}t£z C R, such that \\x\\£ T = (X^-oo WOD < +°° f° r 
r 6 [l,oo) or H^H^ = sup f \x(t)\ < +oo for r = +oo. 

Let be the set of all sequences x E £ r such that x(t) = for t = — 1, —2, —3, .... 

For x € l\ or x 6 £2 , we denote by X = Zx the Z-transform 

00 

X(z) = ^ x{t)z-\ z£C. 

t=—oo 
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Respectively, the inverse Z-transform x = Z l X is defined as 

x(t) = —[ X (e iuJ )e iujt dto, t = 0, ±1, ±2, .... 
If £ € -^2) then X\j is defined as an element of L2(T). 

Let r € Z U {+00} and 9 < r; the case where = —00 is not excluded. We denote 
by £2(9, t) the Hilbert space of complex valued sequences {x(t)}l =e such that ||#||^ 2 (0 )T ) = 

(EI= e W*)l 2 ) 1/2 <+oo. 

Let Zin,oo be the set of all mappings X : T — > C such that X (e^) € L^i—t^-, n) and X (e ltu ) = 
for > 0. Note that the corresponding processes x = Z~ 1 X are said to be band-limited. 

Let Uq : n be the set of all X G Uq jOQ such that there exists a sequence {yk}k=-N € C 2iV+1 
such that X (e lu) ) = J2k=-N Vk^ lku ^\\u\<n}i w bere I is the indicator function. 

We assume that we are given 12 G (tt/2,it), N G Z + U {+00}, sgZ and q < s. The case of 
g = — 00 is not excluded. 

We assume that if N = +00 then g = — 00. 

Let T = {t G Z : g < t < s} if g > -00 and T = {t € Z : t < s} if 9 = -00. 

Let Zjv be the set of all integers k such that \k\ < N if N < +00, and let be the set Z 
of all integers if N = +00. 

Let 3^v be the Hilbert space of sequences {yk}k=-N c ^ provided with the Euclidean norm, 
i.e., such that ||y|| yjv = (J2kez N \yk\ 2 ) < +°°- 

Consider the Hilbert spaces of sequences X = £2 and X- = liifL-, s). 

Let Xn t N be the subset of Af_ consisting of sequences {x(t)}|t £ 7-, where x & X are such that 
x(t) = (Z^X)^) for t G T for some X (e^) G W n ,JV- 

Up to the end of this paper, we assume that the following condition is satisfied. 

Condition 2.1 Either N = +00 or N < +00 and the matrix {sine (kir + Qm)} 1 j? m= _ N is 
nondegenerate. 

Proposition 2.1 Let N < +00, and let £Iq G (vr/2, 7r) 6e selected such that there exists p G (0, 1) 
such that 

min Isinc (wk — flk)\ > p 

k&Z N 

P 

max |sinc (7r/c + Qm)\ < — — for all Q£{£lo,ir). (2.1) 
Then the matrix {sine [kir + Hm)}^ ' m= _ N is nondegenerate for all SI G [Uq,t:). 
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Clearly, (|2.ip holds for any Qq that is close enough to it, since sine (x) — > 1 as x —> and 
sine (x) — > as x — > nm, where m G m ^ 0. Therefore, Condition 12.11 can be satisfied with 
selection of f2 being close enough to tt. 

Proposition 2.2 (%) 7/iV = +oo and q = —oo, then for any x G Xq.n there exists an unique 
X G Un,N such that x(t) = ift<s. 

(ii) If N is finite and s — q > 2N + 1, then for any any x G Xn,N, there exists an unique 
X G Un,N such that x(t) = {Z~ l X)(t). 

(Hi) Assume that N is finite and s — q < 2N + 1. In this case, {x(t)}t^j- G Xq,,n f° r any x G £2 
and any £1 G [f2o,7r). //, in addition, s — q = 2N + 1, then there is an unique X G ^n,iv 
such that x(t) = If s — q < 2N + 1, then there are many X G Uq^n such that 

x(t) = (Z^X)^); they form a linear manifold in X^,N- 

By Proposition I2.2( i). the future {x(t)} t>s of a band-limited process x G Xq is uniquely 
defined by its entire history {x(t), t < s} for any £1 G (0,7r). By Proposition I2.2( ii)-(iii). the 
future of even more "smooth" processes from Xq.^ is uniquely defined by a finite set of historical 
values that has at least 2N + 1 elements for any N < +00 and f2 G [J7o,7r). 

3 Main results 

3.1 Optimal band- limited approximation 

Let x G X be a process. We assume that the sequence {x{t)} t &T represents available historical 
data. Let Hermitian form F : Xq, n x AL — > R be defined as 

s 

F(x,x) = ^ - x (t)\ 2 - 

t=q 

Theorem 3.1 (i) For any N < +00 ; there exists an optimal solution x of the minimization 
problem 

Minimize F(x,x) over i £ I^jv. (3-1) 

(ii) If either N = +00 and s = —00 or N is finite and s — q > 2N + 1, then the corresponding 
optimal process x is uniquely defined. 
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(Hi) If N is finite and s — q < 2N + 1 then there are many optimal processes x; they form a 
linear manifold in Xn,N- 

Remark 3.1 By Proposition \2.'A there exists an unique extrapolation of the band-limited solu- 
tion x of problem $3.1\) on the future times t > s, under the assumptions of Theorem ] 3. lV ii). It 
can be interpreted as the optimal forecast (optimal given ft and N). 

3.2 Optimal sine coefficients 

Up to the end of this section, we assume that either N = +oo or N < +oo, s — q > 2N + 1. 

To solve problem (|3,ip numerically, it is convenient to expand X (e 4aJ ) via Fourier series. 

Consider the mapping Q : — > Xq,n such that x = Qy is such that x(t) = for 
t G (g, s], where 

X{e^)= Y, (3.2) 

kez N 

Clearly, this mapping is linear and continuous. 

Let Hermitian form G : 3V x X— — > R be defined as 

s 

G(y,x) = F(Qy,x) = Y,\x{t)-x{t)\ 2 , x = Qy. (3.3) 

t=q 



Corollary 3.1 There exists an unique solution y of the minimization problem 

Minimize G(y,x) over y € 3^v- (3-4) 



Problem (J3JJ) can be solved via problem (|3,4p ; its solution can be found numerically if 
N < +oo. 

3.3 Solution of problem ( IQil 

Let X be defined by (j3.2fl . where {yu] € 3V- Let x = Z~ 1 X. We have that 



J u \kez N } kez N J u 



-n 



2^ £ Vk ikn/Q + it = n H + ( 3 - 5 ) 

kez N ' kez N 
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We have that 

2 

G(y,*) = ^|x(t)-*(i)| 2 = ]T ° 

t=q t=q 

{y, Ry)y N - 2Re (y, rx) x „ + (px, x)x_ . (3.6) 



— 2/fcsmc (/c7r + fit) — x(t) 

7T * — ' 

kez N 



Here i? : 3^v x 3^v — >■ 3^v is a linear bounded Hermitian operator, r : X- — > is a bounded 
linear operator, p : X- x X— — > X- is a linear bounded Hermitian operator. 

It follows from the definitions that the operator R is non-negatively defined (it suffices to 
substitute x(t) = into the Hermitian form). 

3.4 The case when N < +oo 

Up to the end of this section, we assume that N < +oo and s — q > 2N + 1. In this case, the 
space 3^v is finite dimensional. It follows that the operator R can be represented via a matrix 
R = {Rkm} G C 2N+1 ' 2N+1 , where R km = R mk . In this setting, {Ry) k = J2k=-N R kmVm- 

Theorem 3.2 (i) For any N < +oo, the operator R is positively defined. 

(ii) Problem [3.$ has a unique solution y = R~ l rx. 

( Hi ) The components of the matrix R can be found from the equality 

fi 2 s 

Rkm = —^/ sine (mir + fit)sinc (kir + fit). (3-7) 

TT Z Z — ' 

t=q 

(iv) The components of the vector rx = {{rx) k } k ^ = _ N can be found from the equality 

fi 3 

(rx) k = — } sinc(kn + ttt)x(t). (3.8) 

7T 

t=q 

Corollary 3.2 Let y be the vector calculated as in Theorem \3.2l y = {yk} k= _N- The process 

fi 



x(t) = x(t, q,s) = — 2/fcSmc (kir + fit) 
represents the output of a causal filter that is linear but not time invariant. 
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4 Numerical experiments 



In the numerical experiments described below, we have used MATLAB. 

The experiments show that some eigenvalues of R are quite close to zero despite the fact that, 
by Theorem 13.21 R > 0. Respectively, the error E = \\Ry — rx\\i 2 t q ^ s \ for the MATLAB solution 
of the equation Ry = rx does not vanish. Further, in our experiments, we found that the error 
E can be decreased by the replacing R in the equation x = R~ 1 rx by R £ = R + el, where / is 
the unit matrix and where e > is small. In particular, for e = 0.001, the corresponding error 
E(e) = \\R~ 1 rx — y\\t 2 (q )S ) < rx — y\\g 2 (q jS ), i.e., the approximation for q < t < s is better 
for y = R~ 1 rx calculated for e = 0.001 than for y = R~ 1 rx calculated for e = 0. We have used 
e = 0.001 and N = 15. 

Figures [5 . lM 5 . 2 1 show a example of a process x(t) and the corresponding band- limited process 
x(t) approximating x(t) at times t € { — 25, 15} (i.e, with q = —25, s = 15). 

Figure [5TT1 shows the result for f2 = 0.4; Figure [5721 shows the result for f2 = 1. The values 
of x(t) for t > 15 were calculated using the history {x(s)}_25< s <i5 and can be considered as an 
optimal forecast of x(t). 

We have verified numerically that the matrix {sine (k-ir + ftm)}^ m= _ N is nondegenerate 
in both cases. Therefore, Condition 12.11 is satisfied. In fact, we found that this matrix was 
nondegenerate in all experiments for all kinds of Q and N. 

By Remark 13.11 the extrapolation of the process x € Xq„n to the future times t > s can be 
interpreted as the optimal forecast (optimal given Q and N). 

Remark 4.1 We have used the procedure of replacement R by R e = R + el with small e > to 
reduce the error of calculation of the inverse matrix for the matrix R that is positively defined 
but is close to a degenerate matrix. It can be noted that the same replacement could lead to a 
meaningful setting for the case when e > is not small. More precisely, it leads to optimization 
problem 

N 

Minimize G(y,x)+e 2 \yk\ 2 over y&yN- (4-1) 

k=-N 

The solution restrains the norm of y, and, respectively, the norm of x. 

5 Proofs 

Proof of Proposition \2.1l Let a m ^ = sine (irk + Clm). By (12. ip . there exists p G (0,1) and 
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k G Zn such that 

|ojfc,-fc| = |sinc (— ixk + Qk)\ > p, 

\a m ,k\ = Y \sinc(nk + nm)\ <2N— = p. 

k&Zff, kj^—m k£ZM, kj^—m 

It follows that the matrix {a mj fc}^ k=-N can ^ e transformed into a strictly diagonally dominant 
matrix, i.e., a non-degenerate matrix. □ 

Proof of Proposition \2.2[ The statement of this proposition for N = +00 and q = —00 is 
known in principle. It suffices to consider s = only. Without a loss of generality, we assume 
that s = 0. Further, we assume that T = {t : t < 0}, i.e., it is defined for q = —00. It 
suffices to prove that if x(-) G Afo,jv is such that x(t) = for t G T, then = for i > 0. 
For the sake of completeness, we give below a proof based on Theorem 1 By this theorem, 
processes x(-) € Xn,N are weakly predictable in the following sense: for any T > , e > 0, and 
k G ^oo(0, T), there exists «(•) G ^2(0, +00) n ^oo(0, +00) such that 

\\y-y\\i 2 < e, 

where 

t+T t 

y(t) = n(t — m)a(m), = ?c(t — m)x(m). 

m=f m=— 00 

Let us apply this to a process ir(-) G ^f^v such that = for t G T ■ Let us observe first 
that 

y(t) = Vt < 0. (5.1) 

Let T > be given. Let us show that x(t) = if < t < T. Let {/^(Olitfi be a basis in 
h(~T, 0). Let = Y,m=t ^ ~ m)x(rn). It follows from that = if t < 0. Since 
yi(t) is a continuous function, it follows that yi(t) = for t < 0. It follows that x(t) = if t < T. 

Further, let us apply the proof given above to the function x\{t) = x[t + T). Clearly, 
x\{-) G Xvl,n and X\(t) = for t < 0. Similarly, we obtain that x\{t) = for all t < T, i.e., 
x(t) = for all t < IT . Repeating this procedure n times, we obtain that x(t) = for all t < nT 
for all n > 1. This completes the proof of Proposition 12.21 for N = +00 and q = —00. 

It can be noted that, instead of [4], we could use predictability of band- limited processes 
established in [5]. 

Let us prove the statements (ii) for a finite N. Let us consider first the case when s — q = 
2N + 1. Without a loss of generality, we assume that s = N. It suffices to consider q = —N 



8 



only; in this case, the set T = {t : q < t < s} = {t : — N < t < N}, i..e, T = and it has 
2N — 1 elements. It suffices to prove that if x(-) € <#h,jv is such that x(t) = for t € T, then 
= for t > 0. By (|3~3]) . we have that 

a ^2/fc = 0, -N<t<N, (5.2) 

for some set {y^}- By Proposition 12.11 linear system (15.20 is a system with a non-degenerate 
matrix. Hence ?/& = for all fc. This completes the proof of Proposition 12.21 (ii) for the case 
when s — q = 2N + 1. 

Let us consider the case when s — q > 2N + 1. We assume again that s = N In this case, 
the linear system (|5.2p has considered jointly with the system 

J2 a ^Vk = 0, -q < t < -N. (5.3) 

Clearly, system ()5.2|) - ()5.3[) admits only zero solution again. This completes the proof of Propo- 
sition O (ii) • 

Let us prove statement (hi). Let us consider first the case when s — q = 2N + 1. Since 
homogeneous linear system (|5.2p allows only zero solution for £1 S [^ch 71 ") fo r some £Iq, it follows 
that the non-homogeneous system 

a ^Vk = x(t k ), -N <t<N (5.4) 

admits a unique solution {y^} for any set {x(tf.)}, Therefore, we proved that {x(t)}t<=T £ ^n,N 
for any x G 1%. If s — q < 2N + 1, then there are many solutions of (|5.4|) . and these solutions 
form a linear manifold. This completes the proof of Proposition 12.21 □ 

Proof of Theorem \3.1[ Let us prove statement (i). It suffices to prove that Xn,N is a closed 
linear subspace of ^2(9, s). In this case, there exists a unique projection x of {x(t)}t<=T on Xn,N, 
and the theorem is proven. 

Clearly, for any N < +00, the set £/n,iV is a closed linear subspace of L2(—tt,it). Consider 
a mapping 

Q ■ Z4i,iV Xn,N such that x(t) = (QX)(t) = (Z~ l X){t) for t € T. It is a linear 
continuous operator. By Proposition 12. 2[ it is a bijection. Since this mapping is continuous, it 
follows that the inverse mapping Q^ 1 : Xq n — >• Uq^ is also continuous (see Corollary in Ch.II.5 
[TO] , p. 77). Since the set Lfo.jv is a closed linear subspace of L2(— vr,7r), it follows that Afo.jv is a 
closed linear subspace of X- . This completes the proof of Theorem 13. 1 I f i) . 

Statements (ii)-(iii) follows immediately from Proposition 12.21 This completes the proof of 
Theorem 13.11 □ 
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Proof of Theorem \°J. 6 A Let us prove statement (i). We know that R > 0. Suppose that there 
exists y G C 2N+1 such that y ^ and i?y = 0. Let r* : J^v — > X- be the adjoint operator to the 
operator r* : X- — > 3^v. If r*y 7^ then there exists x G such that G(y, x) < 0, which is not 
possible since G{y,x) > for all y,x. Therefore, r*y = 0, i.e., G(y,x) = (px,x)x_. Further, let 
y be a solution of problem (|3.4p . We have that G(y,x) = G(y + y,x). Hence y + y 7^ y is another 
solution of problem f|3.4j) . This contradicts to Corollary 13.11 that states that this problem has 
an unique solution. Statement (ii) follows from (i) and from classical theory of quadratic forms. 
Statements (iii)-(iv) follow immediately from representation (|3.6 j) . This completes the proof of 
Theorem E2J □ 
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Figure 5.1: Example of x(t) and band-limited process x(t) approximating x(t) for t 6 { — 25, .., 15}, 
with Q = 0.4, and N = 15. The values of x(t) for t > 15 were calculated using {x(s)} s <\5 and can be 
considered as an optimal forecast of x(t). 
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Figure 5.2: Example of x(t) and band-limited process x(t) approximating x(t) for t 6 {—25, .., 15}, 
with = 1, and N = 15. The values of x(t) for t > 15 were calculated using {x(s)} s <i5 and can be 
considered as an optimal forecast of x(t). 
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